Parameter calibration method and apparatus

ABSTRACT

Embodiments of the present invention disclose a parameter calibration method. The method includes: acquiring a calibration template image, where the calibration template image is obtained by photographing a calibration template; performing corner detection on the calibration template image to extract image corners; calculating a radial distortion parameter according to the extracted image corners; performing radial distortion correction according to the calculated radial distortion parameter, so as to reconstruct a distortion correction image; and according to a perspective projection relationship between the calibration template and the reconstructed distortion correction image, calculating intrinsic and extrinsic parameters to implement parameter calibration, where the intrinsic and extrinsic parameters include: a matrix of intrinsic parameters, a rotational vector, and a translational vector. The present invention may be applied to parameter calibration for an imaging apparatus such as a camcorder and a camera in a case of a high distortion.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Patent ApplicationNo. PCT/CN2013/076972, filed on Jun. 8, 2013, which claims priority toChinese Patent Application No. 201210188585.5, filed on Jun. 8, 2012,both of which are hereby incorporated by reference in their entireties.

TECHNICAL FIELD

The present invention relates to the fields of computer vision and imagemeasurement, and in particular, to a camcorder calibration method andapparatus.

BACKGROUND

In an image measurement process and a computer vision application, inorder to determine a relationship between a three-dimensional geometriclocation of a certain point on a surface of a spatial object and acorresponding point thereof in an image, a geometric model for imagingmust be established. Parameters of the geometric model are parameters ofa photography apparatus, such as a camcorder and a camera. Under mostconditions, these parameters can be obtained only by performingexperiments and computations; and a process of solving for theparameters is referred to as camcorder calibration (or cameracalibration). Camcorder calibration is used as an example. Existingcamcorder calibration methods are generally classified into two types:traditional object-based calibration methods and image sequence-basedself-calibration methods.

Among the traditional calibration methods, a two-step method and aplanar template calibration method are typical. The two-step method isto divide calibration work into two steps: First, determine aperspective projection matrix; and then restore intrinsic and extrinsicparameters of a camcorder from the perspective projection matrix.Because a high precision three-dimensional calibration block needs to bemade in this method, it is inconvenient to implement the method. In theplanar template calibration method, according to a characteristic thattwo equations for intrinsic parameters of a camcorder can be establishedbased on calibration points on a same plane, the intrinsic parametersare solved for by using multiple planes of different locations anddirections, and then extrinsic parameters of the camcorder arecalculated. Because it is required to photograph only several planartemplates at different angles or locations in the planar templatecalibration method, an operation is relatively simple. Therefore, thismethod is widely used in practice.

Different from the traditional calibration methods, the self-calibrationmethods do not require a given calibration object but use geometricknowledge of a scene or a constraint relationship of specific camcordermotion to perform calibration on intrinsic and extrinsic parameters of acamcorder. Constraints of intrinsic parameters of a camcorder are mainlyused in these types of methods to restore parameters of the camcorder byusing a method such as solving of Kruppa equations or hierarchicalstep-wise calibration, where the constraints are unrelated to a sceneand motion of the camcorder. However, because the self-calibrationmethods are less precise than the traditional calibration methods, theself-calibration methods are applied only to a given scenario.

On the other hand, distortion modeling and calibration of a camcorderare also extremely important content. In fact, a lens distortion maymore or less exist on a camcorder. There are multiple types ofdistortions on a camcorder, and among those types of distortions, aradial distortion is a main type. For distortion calibration, aclassical method (such as a planar template method) is to first assumethat a camcorder uses a pinhole camera model, obtain intrinsicparameters of the camcorder by performing calibration, and then solvefor a polynomial distortion model parameter by using a non-linearoptimization method. This method is feasible when a distortion of acamcorder is not severe; however, this method fails when it is appliedto a case of a high distortion, such as a fish-eye lens.

It can be learnt that in the prior art, the traditional calibrationmethods fail in a case of a high distortion, and the self-calibrationmethods are less precise than the traditional methods; therefore, how toimplement a camcorder (or a camera) calibration method that is simple tooperate, capable of processing a highly distorted image, and hasrelatively high precision is an urgent problem to be solved.

SUMMARY

Embodiments of the present invention provide a parameter calibrationmethod and apparatus, which can be applied to parameter calibration foran imaging apparatus such as a camcorder (or a camera) in a case of ahigh distortion, and are simple to operate and are of high precision.

According to a first aspect, an embodiment of the present inventionprovides a parameter calibration method, including:

acquiring a calibration template image, where the calibration templateimage is obtained by photographing a calibration template;

performing corner detection on the calibration template image to extractimage corners;

calculating a radial distortion parameter according to the extractedimage corners;

performing radial distortion correction according to the calculatedradial distortion parameter, so as to reconstruct a distortioncorrection image; and

according to a perspective projection relationship between thecalibration template and the reconstructed distortion correction image,calculating intrinsic and extrinsic parameters to implement parametercalibration, where the intrinsic and extrinsic parameters include: amatrix of intrinsic parameters, a rotational vector, and a translationalvector.

Based on a feature of the first aspect, the present invention furtherprovides a parameter calibration method, where the method furtherincludes:

optimizing the calculated intrinsic and extrinsic parameters by using acriterion of a minimum re-projection error and by means of theLevenberg-Marquardt algorithm.

According to a second aspect, an embodiment of the present inventionprovides a parameter calibration apparatus, where the apparatusincludes:

an acquiring unit, configured to acquire a calibration template image,where the calibration template image is obtained by photographing acalibration template;

a detecting unit, configured to perform corner detection on thecalibration template image to extract image corners;

a calculating unit, configured to calculate a radial distortionparameter according to the extracted image corners;

a correcting unit, configured to perform radial distortion correctionaccording to the calculated radial distortion parameter, so as toreconstruct a distortion correction image; and

a calibration unit, configured to, according to a perspective projectionrelationship between the calibration template and the reconstructeddistortion correction image, calculate intrinsic and extrinsicparameters to implement parameter calibration, where the intrinsic andextrinsic parameters include: a matrix of intrinsic parameters, arotational vector, and a translational vector.

With reference to a feature of the second aspect, the present inventionfurther provides a parameter calibration apparatus, where the apparatusfurther includes:

an optimizing unit, configured to optimize the calculated intrinsic andextrinsic parameters by using a criterion of a minimum re-projectionerror and by means of the Levenberg-Marquardt algorithm.

In the method provided in the embodiments of the present invention, inorder to process a highly distorted image, a calibration template imageis first photographed; a radial distortion parameter is estimated byusing a constraint that a straight line in a planar calibration templateis projected as a circular arc in a calibration template image under asingle parameter division model; distortion correction is performed, sothat the calibration template image conforms to perspective projectionimaging; a homography matrix between a reconstructed distortioncorrection image and the planar calibration template is calculated; onan assumption that a principal point is a distortion center and anobliquity factor is zero, an ideal focal length is estimated; and theforegoing result is used as an initial value to perform non-linearoptimization, so as to obtain a precise calibration result. This methodis simple to operate and provides high precision.

BRIEF DESCRIPTION OF DRAWINGS

To describe the technical solutions in the embodiments of the presentinvention more clearly, the following briefly introduces theaccompanying drawings required for describing the embodiments.Apparently, the accompanying drawings in the following description showmerely some embodiments of the present invention, and a person ofordinary skill in the art may still derive other drawings from theseaccompanying drawings without creative efforts.

FIG. 1 is a schematic flowchart of a parameter calibration methodaccording to Embodiment 1 of the present invention;

FIG. 2 is a schematic plan view of a calibration template according toEmbodiment 1 of the present invention;

FIG. 3 is a schematic diagram of polar coordinate transformation of adistortion point (x_(di), y_(di)) in a calibration template image and acorrection point (x_(ui), y_(ui)) in a distortion correction imageaccording to Embodiment 1 of the present invention; and

FIG. 4 is a schematic diagram of a parameter calibration apparatusaccording to Embodiment 2 of the present invention.

DESCRIPTION OF EMBODIMENTS

The following clearly describes the technical solutions in theembodiments of the present invention with reference to the accompanyingdrawings in the embodiments of the present invention. Apparently, thedescribed embodiments are merely a part rather than all of theembodiments of the present invention. All other embodiments obtained bya person of ordinary skill in the art based on the embodiments of thepresent invention without creative efforts shall fall within theprotection scope of the present invention.

Embodiment 1

As shown in FIG. 1, Embodiment 1 of the present invention provides aparameter calibration method, and the method includes the followingsteps:

101. Acquire a calibration template image, where the calibrationtemplate image is obtained by photographing a calibration template.

In step 101, the calibration template adopted in this embodiment may bea calibration template with an array of fixed spacing patterns,specifically including a checkerboard calibration template, acalibration template with an equal spacing solid-circular array, and thelike. Preferably, in this embodiment, a checkerboard calibrationtemplate generally used in a camcorder (or a camera) calibration methodmay be adopted, specifically as shown in FIG. 2.

It should be noted that in order to perform radial distortion correctionand parameter calibration for a camcorder (or a camera), it is requiredto photograph a calibration template, so as to obtain a calibrationtemplate image. In specific implementation, a distribution conditionthat is of grid point coordinates of a calibration template and is on aplane is established according to the number of grid points of thecalibration template in horizontal and vertical directions and a size ofeach grid point.

102. Perform corner detection on the calibration template image toextract image corners.

In step 102, because a distortion generally exists in an image actuallyphotographed by using a lens of a camcorder (or a camera), compared withan actual calibration template, the calibration template image is animage with a distortion. Therefore, the image corners extracted byperforming corner detection are image corners with a distortion.

A person skilled in the art should understand that a corner is animportant feature of an image, which plays an important role inunderstanding and analysis of an image and a graph. There is no explicitmathematic definition for a corner; it is generally considered that acorner is a point with dramatically varying brightness in atwo-dimensional image or a point that has a maximum curvature on an edgecurve in an image. A corner effectively reduces a data volume ofinformation while maintaining important features of an image or a graph,thereby causing a large content of information of the image or graph,effectively increasing a computation speed, facilitating reliablematching of images, and making real-time processing possible. A corneralso plays an extremely important role in the field of computer vision,such as three-dimensional scene reconstruction, motion estimation,target tracing, target identification, and image registration andmatching. Currently, a corner detection algorithm includes grayscaleimage-based corner detection, binary image-based corner detection,profile curve-based corner detection, and the like.

103. Calculate a radial distortion parameter according to the extractedimage corners.

It should be noted that in computer vision, an image is a reflection ofa spatial object in an image plane by using an imaging system, that is,a projection of the spatial object onto the image plane. A grayscale ofeach pixel point in an image represents intensity of reflected light ata certain point on the surface of a spatial object, and a location ofthe pixel point in the image is related to a geometrical location of acorresponding point on the surface of the spatial object. A relationshipbetween these locations depends on a geometrical projection model of acamcorder (or a camera) system, where a projection relationship of anobject from three-dimensional space to an image plane is an imagingmodel. An ideal projection imaging model is central projection inoptics, also referred to as a pinhole model. Under an ideal perspectiveprojection model, a straight line in the calibration template should bealso a straight line in the calibration template image. However, becausea distortion exists in an actual image, under a single parameterdivision model, a straight line in the calibration template is presentedas a circular arc in the calibration template image.

In step 103, the calculating a radial distortion parameter according tothe extracted image corners may specifically include:

103 a. Based on the single parameter division model, model a radialdistortion of a camcorder (or a camera), so as to establish a coordinatetransformation relationship between the calibration template image andthe distortion correction image obtained by correcting the calibrationtemplate image, where specific modeling is shown in formula (1):

$\begin{matrix}{x_{u} = \frac{x_{d}}{1 + {\lambda \; r_{d}^{2}}}} & (1)\end{matrix}$

In formula (1), x_(d)=(x_(d),y_(d)) is a coordinate of any distortionpoint in the calibration template image, x_(u)=(x_(u),y_(u)) is acoordinate of a correction point that is in the distortion correctionimage and obtained after x_(d)=(x_(d),y_(d)) is corrected, λ is a radialdistortion parameter, and r_(d) ²=x_(d) ²+y_(d) ².

103 b. According to a correspondence that a straight line in thecalibration template is presented as a circular arc in the calibrationtemplate image due to an imaging distortion, perform fitting incombination with the image corners to obtain circular arc parameters ofthe circular arc.

Specifically, assuming herein that a non-distortion straight lineequation for a straight line in the calibration template and after anideal perspective projection is performed on the straight line is:

ax _(u) +by _(u) +c=0  (2)

In formula (2), {a,b,c} are parameters of a straight line.

In an actual image, due to existence of the distortion, under the singleparameter division model, the straight line is presented as a circulararc in the calibration template image, and formula (1) is substitutedinto the straight line equation (2) to obtain:

$\begin{matrix}{{x_{d}^{2} + y_{d}^{2} + {\frac{a}{c\; \lambda}x_{d}} + {\frac{b}{c\; \lambda}y_{d}} + \frac{1}{\lambda}} = 0} & (3)\end{matrix}$

It can be learnt from formula (3) that the circular arc already includesinformation about the radial distortion parameter. If the circular arcsare found, the radial distortion parameter may be estimated by using thecircular arc parameters.

Herein, formula (3) is modified as a general form of a circular arc:

x _(d) ² +y _(d) ² +A _(i) x _(d) +B _(i) y _(d) +C _(i)=0  (4)

In formula (4), {A_(i),B_(i),C_(i)|i=1, 2, 3} are circular arcparameters, where

${A_{i} = \frac{a}{c\; \lambda}},{B_{i} = \frac{b}{c\; \lambda}},{and}$$C_{i} = {\frac{1}{\lambda}.}$

For each pixel point belonging to the circular arc, an equation may beobtained. Therefore, to solve for the circular arc parameters in formula(4), at least three pixel points are required to establish an equationset. Because the number of image corners actually extracted is generallygreater than 3, the circular arc parameters in a sense of least-squarecan be obtained by substituting these image corners into formula (4).

103 c. Calculate the radial distortion parameter according to thecircular arc parameters obtained by means of fitting.

Specifically, when there exist three circular arcs, according to thecircular arc parameters {A_(i),B_(i),C_(i)|i=1, 2, 3} obtained by meansof fitting, both the radial distortion parameter λ and a distortioncenter (x_(d0),y_(d0)) may be calculated according to formula (5):

(A ₁ −A ₂)x _(d0)+(B ₁ −B ₂)y _(d0)+(C ₁ −C ₂)=0

(A ₁ −A ₃)x _(d0)+(B ₁ −B ₃)y _(d0)+(C ₁ −C ₃)=0

(A ₂ −A ₃)x _(d0)+(B ₂ −B ₃)y _(d0)+(C ₂ −C ₃)=0  (5)

In the formula above, {A_(i),B_(i),C_(i)|i=1, 2, 3} are three circulararc parameters.

After the distortion center (x_(d0),y_(d0)) is solved for, the radialdistortion parameter λ may be obtained by using formula (6):

$\begin{matrix}{\frac{1}{\lambda} = {x_{d\; 0}^{2} + y_{d\; 0}^{2} + {A_{i}x_{d\; 0}} + {B_{i}y_{d\; 0}} + C_{i}}} & (6)\end{matrix}$

In formula (6), A_(i), B_(i), C_(i) is any one of the three circulararcs.

When the number of circular arcs is greater than 3, a solution to theradial distortion parameter λ in the sense of least-square may becalculated.

104. Perform radial distortion correction according to the calculatedradial distortion parameter, so as to reconstruct a distortioncorrection image.

Specifically, radial distortion correction is performed according to thesolved radial distortion parameter λ; and

according to formula (1),

$\begin{matrix}{{x_{u} = \frac{x_{d}}{1 + {\lambda \; r_{d}^{2}}}}{and}{y_{u} = \frac{y_{d}}{1 + {\lambda \; r_{d}^{2}}}}} & (7)\end{matrix}$

are obtained.

Formula (7) shows a formula in which a coordinate (x_(d),y_(d)) in acalibration template image is directly projected as a coordinate(x_(u),y_(u)) in a distortion correction image after correction.

It should be noted that under such projection, for reasons of integersampling, there may be many unknown information points in a distortioncorrection image. A relatively proper radial distortion correctionmethod is to solve for, in the calibration template image according tothe radial distortion parameter and according to an inverse process thatthe calibration template image is derived from the distortion correctionimage, a coordinate of a distortion point (x_(di),y_(di)) correspondingto a correction point (x_(ui),y_(ui)) in the distortion correctionimage. Bilinear interpolation is performed on the solved coordinate ofthe distortion point (x_(di),y_(di)) in the calibration template image,so as to obtain a coordinate of a correction point (x_(ui)′,y_(ui)′)after radial distortion correction, and so as to implement thereconstruction of the distortion correction image. Herein, a subscript iis a number used for differentiating between different points in a samecoordinate system.

Specifically, in an embodiment of the present invention, the radialdistortion correction may be performed by using the following method,and specific steps are as follows:

1) Move an original point of the calibration template image to thedistortion center (x_(d0),y_(d0)), so as to obtain:

r _(d) ² =x _(d) ² +y _(d) ².

2) For each correction point (x_(ui),y_(ui)) after distortioncorrection, obtain:

$\begin{matrix}{\frac{y_{ui}}{x_{ui}} = {\frac{\frac{y_{di}}{1 + {\lambda \; r_{di}^{2}}}}{\frac{x_{di}}{1 + {\lambda \; r_{di}^{2}}}} = {\frac{y_{di}}{x_{di}} = k_{i}}}} & (8)\end{matrix}$

In formula (8), k_(i) indicates that the distortion center(x_(d0),y_(d0)), the distortion point (x_(di),y_(di)), and thecorrection point (x_(ui),y_(ui)) corresponding to the distortion pointare collinear.

3) In combination with 1) and 2), establish the following equation set:

$\begin{matrix}\left\{ \begin{matrix}{y_{di} = {{\frac{y_{ui}}{x_{ui}}x_{di}} = {k_{i}x_{di}}}} \\{x_{ui} = \frac{x_{di}}{1 + {\lambda \left( {x_{di}^{2} + y_{di}^{2}} \right)}}}\end{matrix} \right. & (9)\end{matrix}$

Solve the equation set to obtain:

$\begin{matrix}{x_{di} = \frac{1 \pm \sqrt{1 - {{4 \cdot \lambda}\; {{x_{ui}\left( {1 + k_{i}^{2}} \right)} \cdot x_{ui}}}}}{2\; \lambda \; {x_{ui}\left( {1 + k_{i}^{2}} \right)}}} & (10)\end{matrix}$

Because λ<0, formula (10) inevitably has two real number solutions;however, in the two solutions, because x_(ui) and x_(di) must be eitherpositive or negative, a valid solution x_(di) can still be uniquelydetermined. After x_(di) is solved for, x_(di) is substituted into thefirst equation of equation set (9) to solve for y_(di).

4) After the distortion point (x_(di),y_(di)) is solved for, a pixelvalue of a point (x_(ui)′,y_(ui)′), after distortion correction isobtained by performing bilinear interpolation.

In another embodiment of the present invention, a distortion point(x_(di),y_(di)) and a correction point (x_(ui), y_(ui)) may further betransformed into polar coordinates for expression. Solving is performedwith the polar coordinates, as shown in FIG. 3, which is specifically asfollows:

Assuming that a distortion point (ρ_(d),θ_(d)) corresponds to acorrection point (ρ_(u),θ_(u)) after correction, θ_(u)=θ_(d), andtherefore it is required to determine only ρ_(d).

According to formula (7),

$\begin{matrix}{\rho_{u}^{2} = {{x_{u}^{2} + y_{u}^{2}} = \frac{\rho_{d}^{2}}{\left( {1 + {\lambda \; \rho_{d}^{2}}} \right)}}} & (11)\end{matrix}$

is obtained.

Then a quadratic equation of one unknown for ρ_(d) ² may be established,and by applying ρ_(d)>0 and a constraint of ρ_(d)<ρ_(u), a uniquesolution to ρ_(d) may be solved for.

105. Calculate intrinsic and extrinsic parameters according to aperspective projection relationship between the calibration template andthe reconstructed distortion correction image to implement parametercalibration, where

the intrinsic and extrinsic parameters include a matrix of intrinsicparameters, a rotational vector, and a translational vector.

According to the perspective projection relationship between thecalibration template and the reconstructed distortion correction image,a homography matrix (Homography) H may be estimated:

s{tilde over (x)} _(u) =H{tilde over (M)}  (12)

In formula (12), s is a scale factor, {tilde over (M)} is a homogeneouscoordinate of a point in the calibration template, and {tilde over(x)}_(u) is a homogeneous coordinate of a point obtained after {tildeover (M)} is projected onto the reconstructed distortion correctionimage, where

H=K[r ₁ r ₂ t]  (13)

where

$k = \begin{bmatrix}f_{a} & c & u_{0} \\0 & f_{b} & v_{0} \\0 & 0 & 1\end{bmatrix}$

is a matrix of intrinsic parameters of a camcorder (or a camera), r₁ andr₂ are rotational vectors and r₁ and r₂ are orthogonal, t is atranslational vector, (u₀,v₀) is a principal point of the matrix ofintrinsic parameters, c is an obliquity factor, and (f_(a),f_(b)) is anideal focal length of a lens of the camcorder (or the camera).

Due to orthogonality of r₁ and r₂,

$\begin{matrix}\left\{ \begin{matrix}{{h_{1}^{T}K^{- T}K^{- 1}h_{2}} = 0} \\{{h_{1}^{T}K^{- T}K^{- 1}h_{1}} = {h_{2}^{T}K^{- T}K^{- 1}h_{2}}}\end{matrix} \right. & (14)\end{matrix}$

In formula (14), h₁ and h₂ are forms of expressing column vectors of thematrix H, and H=[h₁ h₂ h₃]. Formula (14) shows two basic constraintequations that solve for the matrix of intrinsic parameters.

Because only one calibration template image is adopted in the presentinvention, it is impossible to solve for all of 5 unknown numbers in thematrix K of intrinsic parameters. Therefore, several parameters in thematrix of intrinsic parameters are predefined:

a) It is preset that an initial value of the principal point (u₀,v₀)coincides with the distortion center (x_(d0),y_(d0)). Even though it isproven by many researchers that a principal point does not coincide witha distortion center, it is noted that the principal point is generallyextremely close to the distortion center. Therefore, it is proper toassume that the distortion center is the initial value of the principalpoint. A precise coordinate of the principal point is to be obtained bysubsequent non-linear optimization.

b) It is preset that the obliquity factor c=0; and for most lenses, thisis a proper assumption.

Therefore, a solution to a matrix of intrinsic parameters of a camcorder(or a camera) is simplified to a solution to two unknown numbers f_(a)and f_(b). Because of

${{K^{- T}K^{- 1}} = \begin{bmatrix}\frac{1}{f_{a}^{2}} & 0 & {- \frac{u_{0}}{f_{a}^{2}}} \\0 & \frac{1}{f_{b}^{2}} & {- \frac{v_{0}}{f_{b}^{2}}} \\{- \frac{u_{0}}{f_{a}^{2}}} & {- \frac{v_{0}}{f_{b}^{2}}} & {\frac{u_{0}^{2}}{f_{a}^{2}} + \frac{v_{0}^{2}}{f_{b}^{2}} + 1}\end{bmatrix}},$

the constraint of formula (14) is applied to obtain:

$\begin{matrix}{{\begin{bmatrix}m_{11} & m_{12} \\m_{21} & m_{22}\end{bmatrix} \cdot \begin{bmatrix}\frac{1}{f_{a}^{2}} \\\frac{1}{f_{b}^{2}}\end{bmatrix}} = \begin{bmatrix}{{- h_{13}}h_{23}} \\{h_{23}^{2} - h_{13}^{2}}\end{bmatrix}} & (15)\end{matrix}$

In formula (15):

m ₁₁ =h ₁₁ h ₂₁ −u ₀(h ₁₃ h ₂₁ +h ₁₁ h ₂₃)+u ₀ ²(h ₁₃ h ₂₃)

m ₁₂ =h ₁₂ h ₂₂ −v ₀(h ₁₃ h ₂₂ +h ₁₂ h ₂₃)+v ₀ ²(h ₁₃ h ₂₃)

m ₂₁=(h ₁₁ ² −h ₂₁ ²)−2u ₀(h ₁₁ h ₁₃ −h ₂₁ h ₂₃)+u ₀ ²(h ₁₃ ² −h ₂₃ ²)

m ₂₂=(h ₁₂ ² −h ₂₂ ²)−2v ₀(h ₁₂ h ₁₃ −h ₂₂ h ₂₃)+v ₀ ²(h ₁₃ ² −h ₂₃ ²)

Formula (15) is linearly solved to obtain f_(a) and f_(b).

After f_(a) and f_(b) are solved for, in combination with the predefinedprincipal point (u₀,v₀) and the obliquity factor c, the matrix K ofintrinsic parameters may be restored, and then the rotational vector Rand the translational vector t may be solved for.

So far, calibration for geometrical and optical parameters of acamcorder (or a camera) is complete.

The parameter calibration method provided in this embodiment can beapplied to calibration for a camcorder (or a camera) in a case of a highdistortion. In addition, because only one calibration template image isadopted in parameter calibration, compared with an existing camcorder(or a camera) calibration method, this method has advantages, such asbeing simple and effective, and being easy to operate.

Further, in this embodiment, the parameter calibration method mayfurther include the following steps:

106. Optimize the calculated intrinsic and extrinsic parameters by usinga criterion of a minimum re-projection error and by means of the LM(Levenberg-Marquardt, Levenberg-Marquardt) algorithm, so that theintrinsic and extrinsic parameters after optimization become moreprecise.

Specifically, the following objective function is adopted duringoptimization:

$\begin{matrix}{\min {\sum\limits_{j = 1}^{m}\; {{m_{j} - {m\left( {K,R,t,M_{j}} \right)}}}^{2}}} & (16)\end{matrix}$

In formula (16), m_(j) is a coordinate of a point in the reconstructeddistortion correction image, m (K,R,t,M_(j)) represents a coordinate ofa point obtained after a point M_(j) in the calibration template isperspectively projected onto the calibration template image.

When an iteration error is less than a preset threshold, iteration ends,so that a precise matrix K of intrinsic parameters, rotational vector R,and translational vector t of the camcorder (or the camera) areobtained.

In this embodiment, optimization by means of the LM algorithm makesvalues of the intrinsic and extrinsic parameters more precise.

A person skilled in the art should learn that the method provided inthis embodiment may be applied to parameter calibration for an imagingdevice, which includes but is not limited to a camcorder, a camera, andthe like.

Embodiment 2

Based on the calibration method described in Embodiment 1, as shown inFIG. 4, Embodiment 2 of the present invention provides a parametercalibration apparatus, where the apparatus includes:

an acquiring unit 201, configured to acquire a calibration templateimage, where the calibration template image is obtained by photographinga calibration template;

a detecting unit 202, configured to perform corner detection on thecalibration template image to extract image corners; and

a calculating unit 203, configured to calculate a radial distortionparameter according to the extracted image corners.

In this embodiment, the calculating unit 203 specifically includes:

a modeling module 2031, configured to, based on a single parameterdivision model, model a radial distortion according to the followingformula, so as to establish a coordinate transformation relationshipbetween the calibration template image and the distortion correctionimage obtained by correcting the calibration template image:

${x_{u} = \frac{x_{d}}{1 + {\lambda \; r_{d}^{2}}}},$

where x_(d)=(x_(d),y_(d)) is a coordinate of any distortion point in thecalibration template image, x_(u)=(x_(u),y_(u)) is a coordinate of acorrection point that is in the distortion correction image and obtainedafter x_(d)=(x_(d),y_(d)) is corrected, λ is a radial distortionparameter, and r_(d) ²=x_(d) ²+y_(d) ²;

a fitting module 2032, configured to, according to a correspondence thata straight line in the calibration template is presented as a circulararc in the calibration template image due to an imaging distortion,perform fitting in combination with the image corners to obtain circulararc parameters of the circular arc, where a straight line equation inthe calibration template is ax_(u)+by_(u)+c=0, a circular arc equationin the calibration template image is x_(d) ²+y_(d)²+A_(i)x_(d)+B_(i)y_(d)+C_(i)=0, and {A_(i),B_(i),C_(i|i=)1, 2, 3} arecircular arc parameters, where

${A_{i} = \frac{a}{c\; \lambda}},{B_{i} = \frac{b}{c\; \lambda}},{and}$${C_{i} = \frac{1}{\lambda}};$

a calculating module 2033, configured to, according to the circular arcparameters obtained by means of fitting, and according to

(A ₁ −A ₂)x _(d0)+(B ₁ −B ₂)y _(d0)+(C ₁ −C ₂)=0

(A ₁ −A ₃)x _(d0)+(B ₁ −B ₃)y _(d0)+(C ₁ −C ₃)=0

(A ₂ −A ₃)x _(d0)+(B ₂ −B ₃)y _(d0)+(C ₂ −C ₃)=0

solve for a distortion center (x_(d0),y_(d0)) and then calculate theradial distortion parameter in combination with a formula

${\frac{1}{\lambda} = {x_{d\; 0}^{2} + y_{d\; 0}^{2} + {A_{i}x_{d\; 0}} + {B_{i}y_{d\; 0}} + C_{i}}};$

and

a correcting unit 204, configured to perform radial distortioncorrection according to the calculated radial distortion parameter, soas to reconstruct a distortion correction image.

In this embodiment, the correcting unit 204 is specifically configuredto:

according to the radial distortion parameter and according to an inverseprocess that the calibration template image is derived from thedistortion correction image, solve for a coordinate of a distortionpoint (x_(di),y_(di)) in the calibration template image corresponding toa correction point (x_(ui),y_(ui)) in the distortion correction image,where a subscript i is a number; and

perform bilinear interpolation on the solved coordinate, of thedistortion point (x_(di),y_(di)) in the calibration template image, soas to obtain a coordinate of a correction point (x_(ui)′,y_(ui)′) in thereconstructed distortion correction image.

A calibration unit 205 is configured to, according to a perspectiveprojection relationship between the calibration template and thereconstructed distortion correction image, calculate intrinsic andextrinsic parameters to implement parameter calibration, where theintrinsic and extrinsic parameters include: a matrix of intrinsicparameters, a rotational vector, and a translational vector.

In this embodiment, the calibration unit 205 is specifically configuredto:

according to the perspective projection relationship between thecalibration template and the reconstructed distortion correction image,estimate a homography matrix H according to the following formula:

s{tilde over (x)} _(u) =H{tilde over (M)}

where s is a scale factor, {tilde over (M)} is a homogeneous coordinateof a point in the calibration template, {tilde over (x)}_(u) is ahomogeneous coordinate of a corresponding point obtained after {tildeover (M)} is projected onto the reconstructed distortion correctionimage, H=K[r₁ r₂ t],

$k = \begin{bmatrix}f_{a} & c & u_{0} \\0 & f_{b} & v_{0} \\0 & 0 & 1\end{bmatrix}$

is a matrix of intrinsic parameters, r₁ and r₂ are rotational vectorsand r₁ and r₂ are orthogonal, t is a translational vector, (u₀,v₀) is aprincipal point of the matrix of intrinsic parameters, c is an obliquityfactor, and (f_(a),f_(b)) is an ideal focal length;

according to orthogonality of r₁ and r₂ obtain a constraint condition

$\left\{ {\begin{matrix}{{h_{1}^{T}K^{- T}K^{- 1}h_{2}} = 0} \\{{h_{1}^{T}K^{- T}K^{- 1}h_{1}} = {h_{2}^{T}K^{- T}K^{- 1}h_{2}}}\end{matrix};} \right.$

preset that an initial value of the principal point (u₀,v₀) coincideswith the distortion center (x_(d0),y_(d0)), set the obliquity factorc=0, and

obtain the ideal focal length (f_(a),f_(b)) by performing linear solvingin combination with formulas

${K^{- T}K^{- 1}} = \begin{bmatrix}\frac{1}{f_{a}^{2}} & 0 & {- \frac{u_{0}}{f_{a}^{2}}} \\0 & \frac{1}{f_{b}^{2}} & {- \frac{v_{0}}{f_{b}^{2}}} \\{- \frac{u_{0}}{f_{a}^{2}}} & {- \frac{v_{0}}{f_{b}^{2}}} & {\frac{u_{0}^{2}}{f_{a}^{2}} + \frac{v_{0}^{2}}{f_{b}^{2}} + 1}\end{bmatrix}$

and

$\left\{ {\begin{matrix}{{h_{1}^{T}K^{- T}K^{- 1}h_{2}} = 0} \\{{h_{1}^{T}K^{- T}K^{- 1}h_{1}} = {h_{2}^{T}K^{- T}K^{- 1}h_{2}}}\end{matrix};} \right.$

and

restore the matrix of intrinsic parameters and then solve for therotational vector and the translational vector in combination with thepreset principal point (u₀,v₀) and the preset obliquity factor c.

The parameter calibration apparatus provided in this embodiment can beapplied to calibration for a camcorder (or a camera) in a case of a highdistortion; and compared with that in the prior art, the apparatus issimpler to operate because only one calibration template image isadopted in parameter calibration.

Further, in this embodiment, the apparatus further includes:

an optimizing unit 206, configured to optimize the calculated intrinsicand extrinsic parameters by using a criterion of a minimum re-projectionerror and by means of the Levenberg-Marquardt algorithm.

In this embodiment, after the calculated intrinsic and extrinsicparameters are optimized by means of the LM algorithm, values of theintrinsic and extrinsic parameters are more precise.

It should be noted that this embodiment is specific physicalimplementation of Embodiment 1 described above, features of thisembodiment and Embodiment 1 can be cross-referenced. A person skilled inthe art should learn that the apparatus provided in this embodiment maybe applied to parameter calibration for an imaging device, whichincludes but is not limited to a camcorder, a camera, and the like.

A person of ordinary skill in the art may understand that all or a partof the processes of the methods in the embodiments may be implemented bya computer program instructing relevant hardware. The program may bestored in a computer readable storage medium. When the program runs, theprocesses of the methods in the embodiments are performed. The storagemedium may include: a magnetic disk, an optical disc, a read-only memory(Read-Only Memory, ROM), or a random access memory (Random AccessMemory, RAM).

The disclosed are merely exemplary embodiments of the present invention,but are not intended to limit the scope of the present invention.Equivalent variation figured out according to the claims shall fallwithin the protection scope of the present invention.

What is claimed is:
 1. A parameter calibration method, comprising:acquiring a calibration template image, wherein the calibration templateimage is obtained by photographing a calibration template; performingcorner detection on the calibration template image to extract imagecorners; calculating a radial distortion parameter according to theextracted image corners; performing radial distortion correctionaccording to the calculated radial distortion parameter, so as toreconstruct a distortion correction image; and according to aperspective projection relationship between the calibration template andthe reconstructed distortion correction image, calculating intrinsic andextrinsic parameters to implement parameter calibration, wherein theintrinsic and extrinsic parameters comprise: a matrix of intrinsicparameters, a rotational vector, and a translational vector.
 2. Themethod according to claim 1, wherein the step of calculating a radialdistortion parameter according to the extracted image corners comprises:based on a single parameter division model, modeling a radial distortionaccording to the following formula, so as to establish a coordinatetransformation relationship between the calibration template image andthe distortion correction image obtained by correcting the calibrationtemplate image: ${x_{u} = \frac{x_{d}}{1 + {\lambda \; r_{d}^{2}}}},$wherein x_(d)=(x_(d),y_(d)) is a coordinate of any distortion point inthe calibration template image, x_(u)=(x_(u),y_(u)) is a coordinate of acorrection point that in the distortion correction image and obtainedafter x_(d)=(x_(d),y_(d)) is corrected, λ is a radial distortionparameter, and r_(d) ²=x_(d) ²+y_(d) ²; according to a correspondencethat a straight line in the calibration template is presented as acircular arc in the calibration template image due to an imagingdistortion, performing fitting in combination with the image corners toobtain circular arc parameters of the circular arc, wherein a straightline equation in the calibration template is ax_(u)+by_(u)+c=0, acircular arc equation in the calibration template image is x_(d) ²+y_(d)²+A_(i)x_(d)+B_(i)y_(d)+C_(i)=0, and {A_(i),B_(i),C_(i)|i=1, 2, 3} arecircular arc parameters, wherein${A_{i} = \frac{a}{c\; \lambda}},{B_{i} = \frac{b}{c\; \lambda}},{{{{and}\mspace{14mu} C_{i}} = \frac{1}{\lambda}};}$and according to the circular arc parameters obtained by means offitting and according to:(A ₁ −A ₂)x _(d0)+(B ₁ −B ₂)y _(d0)+(C ₁ −C ₂)=0(A ₁ −A ₃)x _(d0)+(B ₁ −B ₃)y _(d0)+(C ₁ −C ₃)=0(A ₂ −A ₃)x _(d0)+(B ₂ −B ₃)y _(d0)+(C ₂ −C ₃)=0 solving for adistortion center (x_(d0),y_(d0)) and calculating the radial distortionparameter in combination with a formula$\frac{1}{\lambda} = {x_{d\; 0}^{2} + y_{d\; 0}^{2} + {A_{i}x_{d\; 0}} + {B_{i}y_{d\; 0}} + {C_{i}.}}$3. The method according to claim 1, wherein the step of performingradial distortion correction according to the calculated radialdistortion parameter, so as to reconstruct a distortion correction imagecomprises: according to the radial distortion parameter and according toan inverse process that the calibration template image is derived fromthe distortion correction image, solving for a coordinate of adistortion point (x_(di),y_(di)) that is in the calibration templateimage and corresponding to a correction point (x_(ui),y_(ui)) in thedistortion correction image; and performing bilinear interpolation onthe solved coordinate of the distortion point (x_(di),y_(di)) in thecalibration template image, so as to obtain a coordinate of a correctionpoint (x_(ui)′,y_(ui)′) in the reconstructed distortion correctionimage.
 4. The method according to claim 3, wherein the step ofcalculating intrinsic and extrinsic parameters according to aperspective projection relationship between the calibration template andthe reconstructed distortion correction image comprises: according tothe perspective projection relationship between the calibration templateand the reconstructed distortion correction image, estimating ahomography matrix H according to the following formula:s{tilde over (x)} _(u) =H{tilde over (M)} wherein s is a scale factor,{tilde over (M)} is a homogeneous coordinate of a point in thecalibration template, {tilde over (x)}_(u) is a homogeneous coordinateof a corresponding point obtained after {tilde over (M)} is projectedonto the reconstructed distortion correction image, H=K[r₁ r₂ t],$K = \begin{bmatrix}f_{a} & c & u_{0} \\0 & f_{b} & v_{0} \\0 & 0 & 1\end{bmatrix}$ is a matrix of intrinsic parameters, r₁ and r₂ arerotational vectors and r₁ and r₂ are orthogonal, t is a translationalvector, (u₀,v₀) is a principal point of the matrix of intrinsicparameters, c is an obliquity factor, and (f_(a),f_(b)) is an idealfocal length; according to orthogonality of r₁ and r₂ obtaining aconstraint condition $\left\{ {\begin{matrix}{{h_{1}^{T}K^{- T}K^{- 1}h_{2}} = 0} \\{{h_{1}^{T}K^{- T}K^{- 1}h_{1}} = {h_{2}^{T}K^{- T}K^{- 1}h_{2}}}\end{matrix};} \right.$ presetting that an initial value of theprincipal point (u₀,v₀) coincides with the distortion center(x_(d0),y_(d0)), presetting the obliquity factor c=0, and obtaining theideal focal length (f_(a),f_(b)) by performing linear solving incombination with formulas ${K^{- T}K^{- 1}} = \begin{bmatrix}\frac{1}{f_{a}^{2}} & 0 & {- \frac{u_{0}}{f_{a}^{2}}} \\0 & \frac{1}{f_{b}^{2}} & {- \frac{v_{0}}{f_{b}^{2}}} \\{- \frac{u_{0}}{f_{a}^{2}}} & {- \frac{v_{0}}{f_{b}^{2}}} & {\frac{u_{0}^{2}}{f_{a}^{2}} + \frac{v_{0}^{2}}{f_{b}^{2}} + 1}\end{bmatrix}$ and $\left\{ {\begin{matrix}{{h_{1}^{T}K^{- T}K^{- 1}h_{2}} = 0} \\{{h_{1}^{T}K^{- T}K^{- 1}h_{1}} = {h_{2}^{T}K^{- T}K^{- 1}h_{2}}}\end{matrix};} \right.$ and restoring the matrix of intrinsic parametersand then solving for the rotational vector and the translational vectorin combination with the preset principal point (u₀,v₀) and the presetobliquity factor c.
 5. The method according to claim 1, furthercomprising: optimizing the calculated intrinsic and extrinsic parametersby using a criterion of a minimum re-projection error and by means ofthe Levenberg-Marquardt algorithm.
 6. The method according to claim 1,wherein the calibration template is a calibration template with an arrayof fixed spacing patterns.
 7. A parameter calibration apparatus,comprising: an acquiring unit, configured to acquire a calibrationtemplate image, wherein the calibration template image is obtained byphotographing a calibration template; a detecting unit, configured toperform corner detection on the calibration template image to extractimage corners; a calculating unit, configured to calculate a radialdistortion parameter according to the extracted image corners; acorrecting unit, configured to perform radial distortion correctionaccording to the calculated radial distortion parameter, so as toreconstruct a distortion correction image; and a calibration unit,configured to, according to a perspective projection relationshipbetween the calibration template and the reconstructed distortioncorrection image, calculate intrinsic and extrinsic parameters toimplement parameter calibration, wherein the intrinsic and extrinsicparameters comprise: a matrix of intrinsic parameters, a rotationalvector, and a translational vector.
 8. The apparatus according to claim7, wherein the calculating unit specifically comprises: a modelingmodule, configured to, based on a single parameter division model, modela radial distortion according to the following formula, so as toestablish a coordinate transformation relationship between thecalibration template image and the distortion correction image obtainedby correcting the calibration template image:${x_{u} = \frac{x_{d}}{1 + {\lambda \; r_{d}^{2}}}},$ whereinx_(d)=(x_(d),y_(d)) is a coordinate of any distortion point in thecalibration template image, x_(u)=(x_(u),y_(u)) is a coordinate of acorrection point that is in the distortion correction image and obtainedafter x_(d)=(x_(d),y_(d)) is corrected, λ is a radial distortionparameter, and r_(d) ²=x_(d) ²+y_(d) ²; a fitting module, configured to,according to a correspondence that a straight line in the calibrationtemplate is presented as a circular arc in the calibration templateimage due to an imaging distortion, perform fitting in combination withthe image corners to obtain circular arc parameters of the circular arc,wherein a straight line equation in the calibration template isax_(u)+by_(u)+c=0, a circular arc equation in the calibration templateimage is x_(d) ²+y_(d) ²+A_(i)x_(d)+B_(i)y_(d)+C_(i)=0, and{A_(i),B_(i),C_(i)|i=1, 2, 3} are circular arc parameters, wherein${A_{i} = \frac{a}{c\; \lambda}},{B_{i} = \frac{b}{c\; \lambda}},{{{{and}\mspace{14mu} C_{i}} = \frac{1}{\lambda}};}$and a calculating module, configured to, according to the circular arcparameters obtained by means of fitting and according to(A ₁ −A ₂)x _(d0)+(B ₁ −B ₂)y _(d0)+(C ₁ −C ₂)=0(A ₁ −A ₃)x _(d0)+(B ₁ −B ₃)y _(d0)+(C ₁ −C ₃)=0,(A ₂ −A ₃)x _(d0)+(B ₂ −B ₃)y _(d0)+(C ₂ −C ₃)=0 solve for a distortioncenter (x_(d0),y_(d0)) and then calculate the radial distortionparameter in combination with a formula$\frac{1}{\lambda} = {x_{d\; 0}^{2} + y_{d\; 0}^{2} + {A_{i}x_{d\; 0}} + {B_{i}y_{d\; 0}} + {C_{i}.}}$9. The apparatus according to claim 7, wherein the correcting unit isspecifically configured to: according to the radial distortion parameterand according to an inverse process that the calibration template imageis derived from the distortion correction image, solve for a coordinateof a distortion point (x_(di),y_(di)) that is in the calibrationtemplate image and corresponding to a correction point (x_(ui),y_(ui))in the distortion correction image; and perform bilinear interpolationon the solved coordinate of the distortion point (x_(di),y_(di)) in thecalibration template image, so as to obtain a coordinate of a correctionpoint (x_(ui)′,y_(ui)′) in the reconstructed distortion correctionimage.
 10. The apparatus according to claim 9, wherein the calibrationunit is specifically configured to: according to the perspectiveprojection relationship between the calibration template and thereconstructed distortion correction image, estimate a homography matrixH according to the following formula:s{tilde over (x)} _(u) =H{tilde over (M)} wherein s is a scale factor,{tilde over (M)} is a homogeneous coordinate of a point in thecalibration template, {tilde over (x)}_(u) is a homogeneous coordinateof a corresponding point obtained after {tilde over (M)} is projectedonto the reconstructed distortion correction image, H=K[r₁ r₂ t],$K = \begin{bmatrix}f_{a} & c & u_{0} \\0 & f_{b} & v_{0} \\0 & 0 & 1\end{bmatrix}$ is a matrix of intrinsic parameters, r₁ and r₂ arerotational vectors and r₁ and r₂ are orthogonal, t is a translationalvector, (u₀,v₀) is a principal point of the matrix of intrinsicparameters, c is an obliquity factor, and (f_(a),f_(b)) is an idealfocal length; according to orthogonality of r₁ and r₂, obtain aconstraint condition $\left\{ {\begin{matrix}{{h_{1}^{T}K^{- T}K^{- 1}h_{2}} = 0} \\{{h_{1}^{T}K^{- T}K^{- 1}h_{1}} = {h_{2}^{T}K^{- T}K^{- 1}h_{2}}}\end{matrix};} \right.$ preset that an initial value of the principalpoint (u₀,v₀) coincides with the distortion center (x_(d0),y_(d0)), setthe obliquity factor c=0, and obtain the ideal focal length(f_(a),f_(b)) by performing linear solving in combination with formulas${K^{- T}K^{- 1}} = \begin{bmatrix}\frac{1}{f_{a}^{2}} & 0 & {- \frac{u_{0}}{f_{a}^{2}}} \\0 & \frac{1}{f_{b}^{2}} & {- \frac{v_{0}}{f_{b}^{2}}} \\{- \frac{u_{0}}{f_{a}^{2}}} & {- \frac{v_{0}}{f_{b}^{2}}} & {\frac{u_{0}^{2}}{f_{a}^{2}} + \frac{v_{0}^{2}}{f_{b}^{2}} + 1}\end{bmatrix}$ and $\left\{ {\begin{matrix}{{h_{1}^{T}K^{- T}K^{- 1}h_{2}} = 0} \\{{h_{1}^{T}K^{- T}K^{- 1}h_{1}} = {h_{2}^{T}K^{- T}K^{- 1}h_{2}}}\end{matrix};} \right.$ and restore the matrix of intrinsic parametersand then solve for the rotational vector and the translational vector incombination with the preset principal point (u₀,v₀) and the presetobliquity factor c.
 11. The apparatus according to claim 7, furthercomprising: an optimizing unit, configured to optimize the calculatedintrinsic and extrinsic parameters by using a criterion of a minimumre-projection error and by means of the Levenberg-Marquardt algorithm.